LIBRARY 



NAVAL . 0,1. . ^"JATlSCHOOI 

MONTEREY, CALlFoRIilA 93940 



NPS 54-82-003 

NAVAI POSTGRADUATE SCHOOL 

Monterey, California 




EVALUATION OF SECJIAVINST 3560.1 TACTICAL 
DIGITAL SYSTEMS DOCUMENTATION STANDARD 
FOR SOFTWARE MAINTENANCE 

Norman ? . Schneidewind 

/ 

February 1982 

Final Report: 1 Jan 30 to 1 Jan 82 



Approved for public release; distribution unlimited. 
Prepared for: 

The Trident Command and Control Systems 
Maintenance Agency 
Newport, Rhode Island 



FedDocs 
D 208 .IM /2 
nPS-5M-82-003 



i 






* Vw 






NAVAL POSTGRADUATE SCHOOL 
Monterey , California 



Rear Admiral J. J. Ekelund 
Superintendent 



David R. Schrady 
Acting Provost 



The work reported herein was supported by the Trident Command and 
Control Systems Maintenance Agency. 



Reproduction of all or part of this report is authorized. 



This report was prepared by: 



DUDLEY KNOX LiBFRARY 
NAVAL POSTGRADUATE SCHOOL 
MONTEREY CA 93943-5101 



Unclassified 



SECURITY CLASSIFICATION OF ThiS PAGE Data Entatad) 



REPORT DOCUMENTATION PAGE 


READ INSTRUCTIONS 
BEFORE COMPLETING FORM 


1. REPORT number 2. GOVT ACCESSION NO. 

NPS54-82-003 j 


3. RECIPIENT’S CATALOG NUMBER 


4. title ''and Subtif/e) 

EVALUATION OF SECNAVINST 3560.1 TACTICAL 
DIGITAL SYSTEMS DOCUMENTATION STANDARD FOR 
SOFTWARE MAINTENANCE 


5. TYPE OF REPORT 4 PERIOD COVERED 

Final Report 
1 Jan 80 to 1 Jan 82 


6. PERFORMING ORG. REPORT NUMBER 


7. AUTHOR^a) 

Norman F. Schneidewind 


9. CONTRACT OR GRANT NUMBERrsJ 


9. PERFORMING ORGANIZATION NAME AND ADDRESS 

Naval Postgraduate School 
Monterey, California 93940 


10. program element. PROJECT. TASK 
AREA 4 WORK UNIT NUMBERS 

N4216680WR00007 


11. CONTROLLING OFFICE NAME AND ADDRESS 

Trident Command and Control Systems Maintenance 
Agency 

Newport Rhode Island 02840 


12. REPORT OATE 

22 February 1982 


13. NUMBER OF P AGES 

27 


14 MONITORING AGENCY NAME 4 ADORESS^/^ from ControlUng Otiice) 


15. SECURITY CLASS, (of this raport) 


1S«. OECLASSl FI CATION/ down GRADING 
SCHEDULE 


16. Distribution st atemen t ro/ 

Approved for public release; distribution unlimited. 


17. DISTRIBUTION STATEMENT (of the abstract an farad /n Block 20, It dlHarant from Raport) 


18. SUPPLEMENTARY NOTES 


19. KEY WORDS (Contlnua on ravarsa aida it nacaaamry and Idantify by block nambar) 

Software maintenance 
Software maintainability 
Software standards 
SECNAVINST 3560.1 
Traceability 


20. ABSTRACT (Contlnua on ravarsa slda It nacassmry and Idantlty by block numbar) 

Management and developers have giv'en insufficient attention to software 
maintenance, the most expensive phase of the software life cycle. Standards 
have improved the ability to develop and design software, but most standards 
do not deal with the maintenance phase in a substantive way. SECNAVINST 
3560.1, Tactical Digital Systems Documentation Standard for Software Mainten- 
ance, was evaluated with respect to its usability for software maintenance. 
Recommendations are made for improving the maintainability aspects of this 
instruction . 



1473 EDITION OF 1 NOV 65 IS OBSOLETE 
S/N 0102-014- 6601 



DD 

1 JAN 73 



SECURITY CLASSIFICATION OF THIS PAGE Datm Sntmrad) 



TABLE OF CONTENTS 



Page No. 



I. INTRODUCTION 2 

II. BRIEF DESCRIPTION OF 3560.1 4 

III. TRACEABILITY 7 

IV. USEFULNESS OF 3560.1 FOR SUPPORTING 12 

SOFTWARE MAINTENANCE 

V. CONCLUSIONS A^JD RECOMMENDATIONS 2 2 

DISCLAIMER 25 

DISTRIBUTION LIST 26 



1 



I. 



INTRODUCTION 



Trident CCSMA has requested the Naval Postgraduate School to evaluate 
the Department of the Navy*s Tactical Digital Systems Documentation 
Standard SEGNAVINST 3560.1 dated 8 August 1974, hereafter referred to as 
3560.1, with respect to its applicability and usefulness for software 
maintenance. This report is divided into the following sections: 

Brief Description of 3560.1 . This section is provided to give the 
reader who is unfamiliar with 3560.1 a brief overview of its 
contents. 

Traceability . Since a major concern of CCSMA is traceability, 
3560.1 is evaluated with respect to its traceability attributes. 
Traceability exists when it is possible to identify the parts of 
the software system, and the corresponding documentation, that 
will be affected by a change in the software stemming from a 
software error correction or enhancement. 

" Usefulness of 3560.1 for Supporting Software Maintenance . Each 
section of 3560.1 that is applicable to software maintenance is 
examined with respect to usefulness for maintenance. 

- Conclusions and Recommendations . Major conclusions concerning 
the adequacy of 3560.1 for software maintenance are drawn and 
recommendations are made to make it more suitable for this 
activity. 

The major conclusion of this report is that, as it stands, 

3560.1 is inadequate for effectively supporting a software 
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maintenance activity. However, with the improvements that have 



been recommended, 
tactical software 



3560.1 could be the equal of recently announced 
standards . 
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II. BRIEF DESCRIPTION OF 3560.1 



1. Tactical Operational Requirement (TOR) 

Tactical digital system functional requirements . 

Serves as a basic software specification for design and program 
implementation . 

2 . System Operational Specification (SOS ) 

- Specific operations desired from application of the digital 
processor supported data system. 

3 . System Operational Design (SOD ) 

- Plan for integrated system program. 

Program functions, core allocations, subprogram definitions and 
interfaces, program data storage plans, and support programs. 

- I/O channel assignments. 

Data to be exchanged between digital processors and peripherals. 
Timing requirements for messages. 

4 . Function Operational Specification (FOS) 

Design of each separate data function. 

- Specify each required action at each operator's position. 

5 . Interface Design Specification (IDS) 

Specifications for interdigital processor message traffic in 
format and content. 

6 . Program Performance Specification (PPS) 

Describes performance requirements for the computer programs 
of the digital processor system. 
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Baseline for configuration control. 

Controlling document for procuring agency. 

7 . Function Operational Design (FOP ) 

Program performance design in operator function terms for 
each operator and peripheral equipment. 

8 . Program Design Specification (PDS ) 

Design details for digital processor programs in programming 
language . 

Used for program maintenance. 

9. Program Description Document (PDD ) 

Design details for each subprogram. 

10 . Data Base Design (DBD ) 

Detailed description of all data items. 

11 . Program Package (PP ) 

Card decks, tapes, listings, etc. 

12. Test Plan (TPL ) 

Defines the scope of tests required to ensure that the system, 
function and program meet all applicable technical, operational, 
and performance specifications. 

- Establishes detailed acceptance criteria for the system and 
identifies each level of testing. 

13. Test Specification (TS) 

Purpose and scope of test. 

Identifies software, hardware, and system to be tested. 

• Test Procedure (TPP ) 

Instructions for test execution and evaluation of results. 
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15 . Test Report (TR ) 



Describes and evaluates discrepancies between program design and 
operation, 

16 . Operator's Manual (OM ) 

Presents procedures for prestandby, operate, monitoring, and 
recovery of the digital processor program. 

Describes the minimal operating environment. 

17 . Program Design Manual (PPM ) 

Provides the theory of combat direction system program processes 
that support the station operator. 

By operator and equipment function, describes the digital 
processor program logic and algorithms that produce actions and 
data displays for the operator. 

18 . Command and Staff Manual 

Provides a nontechnical description of the tactical system. 

- Addresses the mission, characteristics, employment, capabilities 
and limitations of the tactical system. 

System Operator's Manual 

- Sole reference for individual operator duties and station 
function. 

- Explains, at the level required by system operators, every 
control button, switch, readout, and display affected by the 
system program. 
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III. TRACEABILITY 



In order to illustrate the traceability characteristics of 3560.1/ 
TABLE 1 is provided to show the specific references which a given section 
of 3560.1 (e.g., SOS) makes to the other section(s) (e.g. , TOR). Where 
these references occur, an ”X‘* is placed in the appropriate cell of TABLE 
1. The table is read by interpreting the left-hand column as the section 
in which a reference occurs, and the top row, where there is an "X" in 
a cell, to be a referenced section. The resultant matrix indicates the 
degree of traceability that exists in the standard. That is, the density 
or sparseness of the matrix is one measure of the existence or absence of 
traceability , respectively . 

The desired traceability relationship, as implied ty "TDS Documenta- 
tion Relationship,** Figure 2 on page 6 of 3560.1, is shown in FIGURE 1. 
The chart (FIGURE 1) was derived from Figure 2 of 3560.1 by considering 
only those sections of the standard that are concerned with program 
development, design, and testing, The arrows are upward pointing to 
suggest the use of documentation for traceability purposes. For example, 
in FIGURE 1, a Test Report (TR) can be traced to the Tactical Operational 
Requirement (TOR). The actual traceability relationships, as defined by 
TABLE 1 for the docxments shown in FIGURE 1, are shown in FIGURE 2. 
Although a great deal of traceability capability exists in 3560.1, a 
comparison of FIGURE 1 with FIGURE 2 shows that actual traceability is 
less than desired traceability, thus indicating inconsistencies between 
the objectives of the standard, relative to traceability, and the actual 



content of its various sections. 
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Although it is not the conventional approach, the argument can be 
made that not even the "desired traceability relationship" shown in 
FIGURE 1 is complete because traceability, as it is shown, depends on a 
long chain of related documents. For example, the Test Procedure (TPR) 
should be related back to the Tactical Operational Requirement (TOR) , 
which is written in operational user language, if the Test Procedure is 
to be a true reflection of user requirements. In other words, it should 
be easy to see how the Test Procedure meets user requirements, rather 
than trace through several intervening documents in order to discern 
this relationship. In addition, the intervening dociment(s) could con- 
tain errors, which would cause a distorted view of how the TPR is to 
meet user requirements. 



8 



TABLE 1. 

SP ECIFIC THACEABILITY REFERENCES OF 5560.1 



SOH 


X 


CSfl 


X 


y 

Q 

O- 




0 




Qi 


XXX 


TPR 


X 


00 

F— 


X X 


TPL 


XXX 


PP 




Q 

CQ 

Q 


X 


CT 

Q 

Q_ 


X 


00 

Q 

Q_ 


XX XX X 


Q 

0 

U. 


X X 


00 

Q- 

O- 


XXX X 


00 

Q 


X X X 


GO 

0 

U_ 


X XXX 


SOD 


X XX X 


SOS 


XX X X 


CXI 

0 

h— 


XXX X 


il 


CciOOOOOOOOOQOOOOQ- — iGOCciCi^OO^I^i: 
OOOOOCL.OOQCQQ.Q_l — Q.} — OOOOO 
1 — COOOLO — Q.LUQ.Q _0 1 — 1 — Q _000 




< o 



O LU h- • 

O < LU 

CO < u q: 

LU ini — . Z3 - 

Q C-> U- Q h- CO 

< - ^ LU Q: CO LU 

LUQ_ S u U O- 
CO < LU o Q- Qi 
< 2 !-ja.Q:LUOs: 
CQ <Q_cyOQ_cr: I— < 

q: < cr 

< 01 — H-l— h-CiO 
I— OCOCOCOCOLUO 
<CrLULULULUQ-Cli 
QCUl 1 i i OQ_ 



-J < U- CO 



<Qi 

1-0 

OOh- 

< 

Qq: 

2:: LU 

o. 
O 
o 

< LU 

s: H- 

2 : CO 

o>- 

CJOO 






h- 2 : ^ o 

2 : o 1 — - — 

LU -- < 2 : h- 

2 : u o < 

LU< — ,0 • -f— 

q:l-> u_i > 2 : 22 : 

---<ULOOUJ 

3ii-2:ucj — 

0— OLU--LJCO< 2 ) 

LUO^QL-LULULUOCU 

LU coco Q-Cr: -- o 

Q_ LU 000 U_Q 

-JOOQ — i LU 

< < a. LU < O 2 

2 -J -J 2:00 CJ 2: LU O 

o<<o 2 oa_—. 

— • 2: 2 -- z: < --00 }— 

1— ooH*e 32 :i— Q- 

Qi}— }— oicoocroo: 

LU<<LULUU-LU — Cw) 

0- q: q: q-Q c: a. co co 
O LU LUO LUO LU LU 

Q_ Q- LUQ_ OQ 

-lOO 2 0 2 

< O < 2 O 2 : 2 

02:2: — U_<--<< 
— LULU}— Q: 0 i!— QiCi 

1— I—}— OLUOOOO 
OC0C02K-O2OO 

<>->-320:2)0:0: 

' -U_ 



CiiOOOOOOOGO 000 o 
1 — coooLL . — 1 n.n. 



9 




FIGURE 1. DESIRED TRACEABILITY RELATIONSHIP. 
* MISSING IN FIGURE 2. 
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FIGURE 2. ACTUAL TRACEABILITY RELATIONSHIP BASED ON 
REFERENCES AMONG SECTIONS. 
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IV. USEFULNESS OF 3560.1 FOR SUPPORTING SOFTOARE MAINTENANCE 



Traceability, which was covered in Section III, is a capability 
which is useful for both software development and maintenance. In this 
section, a sepcific examination is made of the adequacy of 3560.1 for 
software maintenance. Each section of 3560.1 that either mentions main- 
tenance or could be useful for software maintenance is analyzed below. 
Recommendations for improvements in the coverage of software maintenance 
are noted. Section and page references are to 3560.1. 

TABLE 2 is provided to summarize those sections of 3560,1 that are 
applicable to software maintenance. The table shows section, section 
number and page references, purpose of the section, and an indication of 
whether the section is adequate for software maintenance. Brief remarks 
are given, where appropriate. In the case of negative remarks, expla- 
nations are provided following TABLE 2, 

Tactical Operational Requirement Section 1.5.4 Test Programs , 

P. 1-13 . 

Reference is made to any maintenance programs that might be 
required. The paragraph seems to refer to software that is used to 
maintain hardware. Specific reference should be made to the need to 
provide software tools for maintaining software in addition to the 
software used for maintaining hardware. 

2 . Interface Design Specification Section 1.0 Purpose, p. 2-41 . 

This section states: "Upon completion of program development, 

the Interface Design Specification shall serve as a joint life cycle 
configuration control device for digital processor program maintenance 
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of the interface.’* Except for the section noted in TABLE 2 and the 
corresponding explanation of remarks, the IDS contains good sections 
for supporting software maintenance. This is due primarily to the 
thoroughness of treatment of items, such as signal definitions and inter - 
processor communications. 

3 . Program Performance Specification Section 1.0 Purpose, p. 2-83 . 

This section states: "The Program Performance Specification shall 

describe in detail all the operational and functional requirements 
necessary to design, test, and maintain the required digital processor 
program." As shown in TABLE 2 and the explanation of remarks, there are 
sections of the PPS which are redundant, unclear, or require more empha- 
sis on validation as opposed to verification. With respect to the last 
item, see the following section on the Program Description Document, 
which contrasts validation with verification. The comments in that 
section apply as well to the PPS. For these reasons the PPS is not 
entirely adequate for software maintenance. 

4 , Program Description Document Section 1.0 Purpose, p. 2-137 . 

This section states: "As a detailed compendium of the subprogram 

structure, the Program Description document will ser^/e as the essential 
instrument for subsequent use by operational, maintenance , and contractor 
personnel diagnosing troubles, making adaption changes, designing and 
implementing modifications to the system , and in introducing or adding 
new subprogram functions to the completed program." Thus, the PDD is 
one of the major documents which govern the conduct of software mainte- 
nance. As shown in TABLE 2 and in the corresponding explanation of re- 
marks, the quality assurance provisions of the PDD provide sufficient 
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emphasis on verifying programs , via testing, but insufficient emphasis 
on validating programs against tactical requirements (e.g, , TOR) , As 
defined by Lewis: ’’Verification is the iterative process aimed at 

determining whether the product of each step in the software develop- 
ment cycle fulfills all the requirements levied by the previous step: 

Does B fulfill requirements of A? Does C fulfill requirements of B, and, 
implicitly, fulfill requirements of A? ... . Validation is essentially 
that part of verification and validation which looks back at the software 
requirements and determines through testing that they are (or are not) 
satisfied by the observable system performance indicators. The impli- 
cation of validation is that the system will meet its operational life 
cycle design commitments.”^ Thus the FDD is not entirely adequate for 
software maintenance, although it does contain many comprehensive and 
detailed sections. 

5 . Program Package Section l.Q Purpose, p. 2-165 . 

This section states: ”The Program Package document shall con- 

sist of all the program material items necessary for the procuring agency 
to produce, maintain , and update the digital processor program.” Several 
sections of the PP refer to card decks and magnetic tapes as the media for 
source programs. These sections should be augmented to allow for the 
possibility of other physical media, such as disc pack, cassette, 
cartridge, and ROM. In addition, the possible use of interactive compila- 
tion and debugging facilities for software production and the storing 



Robert 0. Lewis, ’’Software Verification and Validation,” in 
John D. Cooper and Mathew Fisher (Editors) , Software Quality Management , 
Chapter 15. 
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of program files in a time-sharing facility should be recognized. Finally, 
the document should allow for the possibility of computer to computer 
transfer of program files, without the intervening physical media of 
cards and tapes. 

6 . Operator's Manual Section 1.0 Purpose, p. 4-5 . 

This section states: "The Operator ''s Manual shall be limited to 

instructions for preparing and maintaining the digital processor program 
in the required state of capability in order that the operational 
mission may be accomplished." As shown in TABLE 2, there are no sections 
of the OM that are considered inadequate for software maintenance. 
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TABLE 2 



SEC 

FOS 

PPS 

PDD 

SOM 



FOS 

PPS 



FOS 

PPS 

PDD 

TS 



PPS 

PPS 

PPS 

PPS 

PDS 

PDS 

PDS 



*Y/N 



SUMMARY OF 3560,1 SECTIONS 
APPLICABLE TO SOFTV/ARE MAINTENANCE 

INPUTS 



SEC # 


PAGE 


PURPOSE 


*Y/N 


REMARKS 


3,3.1 


2-37 


CHARACTERISTICS 


N 


UNCLEAR 


3.4.N.1 


2-95 


CHARACTERISTICS 


Y 




3.4 


2-146 


FORMATS 


Y 




N 


4-38 


DATA ENTRY 
PROCESSING 


Y 




3,3.2 


2-37 


PROGRAM PROCESSING 


Y 




3,4,N,2 


2-98 


FUNCTION PROCESSING 
OUTPUTS 


Y 




3.3,3 


2-37 


DISTRIBUTION 


Y 




3,4,N,3 


2-99 


CHARACTERISTICS 


Y 




3,4 


2-146 


FORMATS 


Y 




3.2,4 


3-29 


TEST OUTPUTS 
FUNCTIONS 


Y 




3,3 


2-90 


FUNCTION DESCRIPTION 


N 


REDUNDANT 


3,3.5 


2-92 


FUNCTION DESCRIPTION 


N 


REDUNDANT 


3,4 


2-95 


FUNCTION REQUIREMENT 


N 


REDUNDANT 


TABLE 3-11 


2-97 


FUNCTION VS. STATE 


Y 


GOOD TABLE 


3,1 


2-123 


FUNCTION REQUIREMENT 


Y 




3,2 


2-124 


FUNCTION DESCRIPTION 


Y 




3,4 


2-127 


FUNCTION FLOW 


Y 





COLUMN; "Y" MEANS SECTION IS ADEQUATE FOR SOFTWARE MAINTENANCE. 

”N’* MEANS SECTION IS INADEQUATE FOR SOFTWARE MAINTENANCE. 
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DATABASE 



SEC 


SEC # 


PAGE 


PURPOSE 


Y/N 


REMARKS 


PPS 


3.5 


2-99 


REQUIREMENTS 


N 


UNCLEAR 


PDD 


3.3.2 


2-145 


SUBPROGRAM DATA DESIGN 


N 


TERMINOLOGY 


DBD 


ALL 


2-153 


COMMON DATA DESCRIPTION 
INTERFACES 


Y 




SOS 


5.2 


2-14 


PERIPHERAL SYSTEM 


N 


VAGUE 


SOS 


5.3 


2-15 


OPERATOR 


N 


VAGUE 


SOS 


5.4 


2-15 


INTERSYSTEM 


Y 




SOD 


5.2 


2-28 


PERIPHERAL SYSTEM 


Y 




SOD 


5.3 


2-28 


OPERATOR 


N 


INCOMPLETE 


SOD 


5.4 


00 

CM 

(N 


INTERSYSTEM ON-LINE 


Y 




IDS 


1.0 


2-41 


INTERFACE REQUIREMENTS 


N 


CONFUSING 


IDS 


2. 0-8. 3 


2-42 


INTERFACE REQUIREMENTS 


Y 


GOOD SECTIONS 


PPS 


3.2.3 


2-89 


INTERFACE I.D. 


Y 




PPS 


3.3.3 


2-92 


DIG. PROC. BLOCK DIA. 


Y 




PPS 


3.3.4 


2-92 


PROGRAM 


Y 




PDD 


3.8 


2-149 


SUBPROGRAMS 


Y 




PDD 


FIG. 3-2 


2-150 


SUBPROGRAMS 

TESTING 


Y 




PPS 


3.4.N.4 


2-99 


SPECIAL REQUIREMENTS 


Y 




PPS 


4.2 


2-101 


TEST REQUIREMENTS 


Y 




FOD 


5 


2-115 


TEST & SIM. SCENARIOS 


Y 




TPL 


1.0 


3-5 


SCOPE & LEVELS 


N 


TERMINOLOGY 
VALIDATION NEEDED 


TPL 


1.1.2 


3-7 


EQUIPMENT 


Y 




TPL 


1.1.3 


3-8 


SUPPORT SOFTWARE 


Y 




TS 


3.2 


3-23 


SYS. /PROG. DEFINITION 


Y 




TPR 


1.0 


3-35 


INSTRUCTIONS & EVAL. 


N 


UNCLEAR 


TPR 


3.2.1 


3-40 


EQUIPMENT PREPARATION 


Y 




TPR 


3.2.3 


3-41 


TEST PROCEDURE 


N 


UNCLEAR 


TPR 


7 


3-43 


SUPPORT SOFT. REQUIRE. 


Y 




TR 


ALL 


3-45 


TEST REPORTS 


Y 


EXCEPT 



PATCH REFERENCE 
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QUALITY ASSURANCE 



SEC 


SEC # 


PAGE 


PURPOSE 


Y/N 


REMARKS 


PPS 


4. 


2-100 


QA PROVISIONS 


N 


VALIDATION NEE 


PDD 


4. 


2-149 


QA PROVISIONS 


N 


VALIDATION NEE 


TPL 


9. 


3-14 


EVALUATE TEST RESULTS 


N 


LOOSE 


TS 


9. 


3-26 


SPECIFY TEST REPORTS 


N 


VALIDATION NEE 






SUBPROGRAMS & SUBROUTINES 






PDS 


3,4.3 


2-131 


REFERENCE CONTROL 


Y 




PDS 


6 . 


2-133 


COMMON SUBROUTINES 


Y 




PDD 


1.0 


2-137 


SUBPROGRAM STRUCTURE 


Y 




PDD 


3.5 


2-148 


LIBRARY SUBROUTINES 


Y 




PDD 


3.£ 


2-148 


INITIATION 


Y 










PROGRAMMING 






SOD 


6.1 


2-29 


STANDARDS 


Y 




PDS 


3.3 


2-127/8 


MEM. & PROC. ALLOC. 


Y 




PDS 


3.5 


2-131 


LANGUAGE 


Y 








CAPACITY REQUIREMENTS 






FOS 


5.2.2 


2-39 


CONSOLE CAPACITY 


Y 




PPS 


3.5 


2-100 


PROGRAM CAPACITY 


Y 








CONFIGURATION MANAGEMENT 






SOS 


4.2 


2-14 


PHYSICAL CONFIGURATION 


Y 




PDS 


3.5 


2-132 


PROGRAM CONFIGURATION 


Y 




OM 


3.1 


4-8 


MINIMUM CONFIGURATION 


Y 




PDM 


3 


4-17 


SUBPROGRAM CONFIGURATION 


Y 










OPERATION 






OM 


4.3 


4-9 


PARAMETER VALUES 


Y 




OM 


6 


4-9 


TROUBLE REPORTS 


Y 




OM 


7 


4-10 


RECOVERY 


Y 
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INPUTS 



Section 3.3.1 (p. 2-37) of FOS reads: "Data type by source, its period- 

icity of update rate, and expected and/or reliability will be provided 
in this paragraph." The meaning of the underlined words is unclear. 



FUNCTIONS 

Sections 3.3 (p. 2-90), 3.3.5 (p. 2-92) and 3.4 (p. 2-95) of PPS overlap 
to some extent. 



DATABASE 

Section 3.5 (p. 2-99) of PPS reads: "Adaptation data is that data that 

can be centrally modified as needed to define the scope of operational 
functions within prescribed limits." The meaning of this statement is 
unclear . 

Section 3.3.2 (p . 2-145), Variables of PDD refers to "program" in line 2 
and "constant" under "a. constant name." The word "variable" was 
probably intended in both cases. 



INTERFACES 

Sections 5.2 (p. 2-14) and 5.3 (p. 2-15) of SOS provide insufficient 
information concerning what comprises a peripheral systems interface 
and operator interface, respectively. 

Section 5.3 (p. 2-28) of SOD references a Peripheral System Interface 
section 5.2 rather than spelling out what is requred for the Operator 
Interface. 
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INTERFACES (CONTINUED) 



Section 1.0 (p.2-41) of IDS reads: "This specification establishes a set 

of requirements for the preparation of a document for defining the design 
interdigital processor digital interfaces . " The underlined portion is 
confusing . 



TESTING 

Section 1.0 (p. 3-5) of TPL reads: "The test plan shall define the scope 

of tests required to ensure that the system, function, and/or program 
meets all applicable technical , operational and performance specification." 
Since the only categories of speciation in 3560.1 are "operational," 
"performance," and "design," the meaning of "technical" is not clear. 

Section 1.0c (p. 3-6) of TPL (Function Test) and Section l.Od (p. 3-6) 
should stipulate validation against the TOR in addition to verification 
against performance and design specifications and program description 
document , respectively . 

Section 1.0 (p. 3-35) of TPR reads: "Test procedures provide for the 

quantitative results of tests, which are later extracted for the tests 
themselves." The meaning of this statement is not clear. 

Sections 3.2.3j (p. 3-42) of TPR reads: "The procedure must agree 

exactly with the program behavior." The meaning of this statement is 
not clear. 
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TESTING (CONTIITUED) 



Sections 1.1.1 (p. 3-45) and 1, (p, 3-49) of TR imply the allowance of 

patches in programs during testing. This is believed to be a poor 
practice . 



QUALITY ASSURANCE 

Section 4. (p. 2-100) of PPS , Section 4. (p, 2-149) of PDD, and Section 

9. (p. 3-26) of TS should stipulate validation against the TOR in addition 

to program verification. 

Section 9. (p. 3-14) of TPL should contain a statement that government 

personnel must witness the tests. Although this section, as written, is 
concerned with procurement rather than maintenance , the argument for 
using government witnesses is valid for maintenance. 
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V. CONCLUSIONS AND PS COMMENDATIONS 



Based on the foregoing analysis, the following conclusions and 
recommendations are presented relative to the usability of 3560,1 for 
software maintenance. 

1. The standard is comprehensive and detailed. Considering the fact 
that it was issued in 1974, it was unable to benefit from the hindsight 
that is expressed in this report, which is based mainly on advances that 
have been made in programming methodology since 1974. If 3560.1 were 
updated to reflect new programming technology, it could be a more com- 
prehensive standard than MIL-STD 1679. Notable aspects of the standard 
are the following: 

Applicable Documents statements. 

Resource budgets (time, space). 

Numerous examples . 

Content Check Lists. 

Interfaces descriptions. 

- Test coverage. 

Detailed Table of Contents for each specification. 

2. As pointed out in Section III, there are some deficiences in the 
vital area of traceability. 

3. As demonstrated in Section IV, there should be more emphasis on 

validation. To quote from page 14 of 3560.1: **The Tactical Operational 

Requirement shall serve as a life cycle configuration management device 
for specifying the overall tactical operational software capability 
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requirements for the combat system." That being the case, there should 
be more emphasis on validating against the TOR, with respect to QA 
procedures , during both development and maintenance . 

4. There are many examples of redundancies and use of inconsistent 
terminology in the standard. Examples of the former are: 

Section 7. Data Unit Descriptions, p. 2-57 and Section 8. 
Message Descriptions, p. 2-69 of Interface Design Specification 
contain similar material. 

Test specification System, op. 3-20 to 3-26 and Test Specifica- 
tion Function, pp. 3-27 to 3-31 contain similar material. 

- Much of the material in the Test Plans and Test Specifications 
is similar. These could be combined into a single document, 
with resultant reduction in verbage and increase in clarity. 
Each of the documents is described in the format of purpose, 
scope, typical content, etc. followed by the actual foinnat of 
document content. Much of the material is duplicated between 
the two sections (e.g.. Test Procedxires pp. 3-35 to 3-37 and 
pp. 3-39 to 3-42) . This material could be combined in many 
of the documents . 

Examples of inconsistent terminology are: 

Section 1.0 Purpose, p. 2-153 and Section 9. Notes, p. 2-162 
of Data Base Design refer to a Subprogram Description Document. 
This document is not defined or described in 3560.1 by that 
name. 

Section 1.1.3 Analysis, p. 3-46 of Test Reports refers to 
Operational/Functional Specification. This specification is 
not defined or described in 3560.1 by that name. 
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5. The standard is much more oriented to software development than 
to software maintenance. It also seems to have a strong orientation to 
the Navy Tactical Data System. A more general orientation might be 
preferable in order to achieve wider applicability to a variety of 
software systems . 

6. It would be useful for both software development and maintencince 
activities to provide a section in the standard that describes the 
material in subject matter instead of document format, i.e., a descrip- 
tion of all document sections that refer to inputs, all that refer to 
outputs, etc. Such a breakdown was used in TABLE 2 of this report. 

7. In summary, an inexperienced programmer would probably have 
difficulty applying 3560.1 to maintenance because of the problems 
mentioned in this report. Because of the great shortage of skilled 
software personnel, the criterion of usability of the standard by an 
inexperienced programmer is appropriate . 
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DISCLAIMER 



The opinions expressed in this report are strictly those of the 
author and do not necessarily reflect the opinions of the Naval 
Postgraduate School, Department of the Navy, or Department of Defense. 
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